Search

Research methodologies in several of the social sciences require the use of tests. When assisting social science researchers who seek information on tests, reference librarians do not, themselves, always have direct access to the instruments. Librarians should not only have the knowledge that various print and electronic database resources are available to gain needed information, but they should also have a strategy by which these sources can be searched in an expedient and thorough manner. This paper offers a search strategy that can be used with the materials commonly found in a large research or specialized test library. The search strategy is presented in a flow chart that suggests test bibliographies and reviews for published instruments and other sources for unpublished experimental measures. (Contains the flowchart and 48 references.) (Author/SLD)

The validity of criterion-referenced interpretations of the proficiency scales of the National Assessment of Educational Progress (NAEP) is discussed. A major goal of the NAEP scales is to describe student achievement in specific content areas from grade 3 (age 9 years) through grade 11 (age 17 years). The numerical values for NAEP scales are intended to be interpreted almost exclusively from a criterion-referenced perspective. The NAEP intends to accomplish the goal of making criterion-referenced interpretations of examinee scores through the use of item response theory methodology. The basis for classifying an item at a proficiency level is the estimated probability of success of examinees at different proficiency levels. The implications for criterion-referenced interpretations are discussed in the context of the Science Proficiency Scale (SPS). Problems with interpreting SPS results are due to the following factors: (1) the science domain is not well defined; (2) it is doubtful that learning in this domain proceeds through an orderly sequence; (3) the number of test items that could be developed is virtually unlimited; and (4) many problems are generated by the interactions between the examinee's past experiences and item content. The NAEP mathematics, reading, and history scales are also reviewed. It is suggested that the NAEP proficiency scales do not yield valid criterion-referenced interpretations. Four figures illustrate the discussion. A 27-item list of references is included. (SLD)

Experimenters sometimes wish to estimate for a particular dependent variable the proportion of total group variance that is associated with mean differences among fixed treatments or subject classifications. Hays (1981) represents this proportion by the parameter omega squared. A point estimate may be easily computed as a function of the number of treatments, the total sample size, and the mean squares between and within treatments. This paper presents tables which facilitate the construction of 90 or 95 percent confidence intervals for omega squared. (Author)

In two studies, children rated infant pictures on an "unhappy-happy" scale after being told how adolescent male models (black or white) had allegedly rated them. The subjects in Experiment 1 were black and white females and males (ages 9-12 years) attending inner-city schools in Des Moines, Iowa. Those in Experiment 2 were white females and males (ages 10-11 years) attending school in a predominantly white, middle-class, university community (Iowa City). In both experiments, the subjects as a total group were significantly affected by the models' ratings. In Experiment 1, girls conformed more than did boys; there was no sex difference in Experiment 2. With regard to race, only one obtained effect is of any direct interest (in Experiment 1, black subjects conformed more to white than to black models). The results suggest that social-class differences may now exist in the conformity behavior of the sexes. The general absence of race effects is viewed as encouraging, from an equalitarian standpoint. (Author)

Multiple-choice standardized achievement tests of English vocabulary and reading comprehension and of mathematics were administered to samples of 592 grade eight students and 615 grade five students. Two forms of each test unit were prepared. The control groups took forms containing items with four responses, while the experimental groups took forms which had an additional response of I don't know. A few fictitious vocabulary items having no right answers were included in each of the English test units. In grade eight tests the mean scores of the control groups were higher. For the grade five samples there were no differences in mathematics; and differences in English (the control group obtaining higher scores) were found only for low ability students. Item discrimination indices obtained from the two forms did not show any significant differences. There was a negative linear relationship between percentage choosing the I don't know response and percentage correct. In general, lower ability students used the response more often than those with higher ability excePt in the case of the fictitious items. Sex differences were also found to be a factor in the use of the I don't know response. (Author/MV)

A brief discussion of the concept of validity, a description of the nature and purposes of the Iowa Tests of Educational Development (ITED), and a rationale for the inclusion of ITED results as part of the overall program evaluation of a secondary school are presented. According to the authors, the ITED can validly be used as one of the data-gathering instruments for program evaluation if the evaluation instruments are judged on the basis of the behaviors that they require of students. (Author/MV)

A statewide survey of ITED testing programs was conducted to provide more useful materials to test users, better services to Iowa high schools, and to enable high schools to improve the effectiveness of their ITED programs. A two-part questionnaire was developed and sent to the testing directors of 429 public and private high schools in November, 1980 and January, 1981. Conclusions drawn from the survey results were that Iowa high schools are generally satisfied with ITED testing programs. There were indications that student motivation for taking tests could be better. It is believed that a major factor influencing student motivation is the attitude of teachers toward the tests. Testing directors who would like their teachers to have a more positive attitude toward testing should consider adopting a method to distribute ITED scores to teachers as soon as scores become available, providing help to the teachers in the interpretation and use of scores, and informing teachers regarding the administrators' and counselors' use of test scores. (PN)

Rigorous comparison of the reliability coefficients of several tests or measurement procedures requires a sampling theory for the coefficients. This paper summarizes the important aspects of the sampling theory for Cronbach's (1951) coefficient alpha--a widely used internal consistency coefficient. This theory enables researchers to test a specific numerical hypothesis about the population alpha and to obtain confidence intervals for the population coefficient. It also permits researchers to test the hypothesis of equality among several coefficients, either under the condition of independent samples or when the same sample has been used for all measurements. The procedures are illustrated numerically, and the assumption and derivations underlying the theory are discussed. (Author)

When a program evaluation is conducted on the basis of gains made in standardized achievement test scores between a pretest and a posttest, serious bias can result where a floor effect exists on one or both of the test levels. Based upon the expanded standard scores on the Comprehensive Tests of Basic Skills from three successive regular spring test administrations and a special mid-period out-of-level test administration, this study examined score distributions for schools in a large city school system to determine if their compensatory program evaluations were biased by a floor effect. The distributions were examined for changes in location, variation, and skewness as students moved across time, taking different levels of the test at different times. The pattern of distributional characteristics for both reading and mathematics over the four testing periods indicated that a floor effect was quite unlikely to bias group comparisons in the upper elementary grades under study (grades 5 and 6). (Author/CP)

Measures of achievement in reading and mathematics following implementation of different curricular programs for disadvantaged pupils in the New Orleans system are compared. Programs implemented in Grades 4 through 7 include: (1) no compensatory programs; (2) traditional compensatory programs; (3) PLAN (Programmed Learning in Accordance with Needs) as a compensatory program. Pupils in each group were administered the Comprehensive Tests of Basic Skills during March of the years 1976, 1977 and 1978. A comparison of the program revealed that (1) both compensatory programs produce achievement scores as high as or higher than those produced by the schools without compensatory programs for pupils of the same initial achievement; (2) pupils of initial ability above the average for compensatory eligibility programs do unusually well when PLAN is begun early in their school careers; (3) the effect of the PLAN type program appears to be cumulative in mathematics with pupils of lower ability; (4) pupils in compensatory programs may fall back in mathematics during seventh grade. Relevant tables and graphs are appended. (RL)

The readability level of passages from three subtests of the Iowa Tests of Educational Development (ITED), Forms X-6 and Y-6, were compared with the readability level of passages selected from the Des Moines Resister, Reader's Digest, Time, Newsweek, Saturday Review, and 18 high school textbooks from the fields of social studies, science, and language arts and literature. The three ITED subtests analyzed were Concepts and Reading Materials in the Social Sciences, Background and Reading Materials in the Natural Sciences, and Literary Materials. Four readability formulas were used to determine the readability levels: the Dale-Chall formula, the Flesch formula, Gunning's FOG Index of Readability, and McLaughlin's SMOG Readability Index. Results indicated that the readability levels of the passages in the ITED social studies, science, and literary subtests were comparable to materials which high school students are likely to encounter both in and out of the classroom. (BW)

It is proposed that statistical tests for item bias, when they are based on the three-parameter logistic item characteristic curve, should begin by testing for differences in the third parameter, the pseudo-chance level or "guessing" parameter. Other item-bias studies that have not considered the third parameter are discussed. Asymptotic significance tests for the three-item parameters are described. An experiment is described based on the 60-item Reading Comprehension Subtest of the Tests of Achievement and Proficiency, using the results for 400 black and 400 white ninth-grade students. None of the 60 items showed any indication of bias in the pseudo-chance level parameter so a weighted average parameter was computed and used in the remainder of the study. Comparisons of the difficulty and discrimination parameters did indicate some biased items. Some interpretation of the results are provided in relation to the content of the items. (Author/CTM)

The technical problems involved in obtaining Bayesian model estimates for the regression parameters in m similar groups are studied. The available computer programs, BPREP (BASIC), and BAYREG, both written in FORTRAN, require an amount of computer processing that does not encourage regular use. These programs are analyzed so that the performance of algorithms might be improved to a point where inclusion in the CADA Monitor would be feasible. Slight modifications to the preparatory program BPREP, and major changes to BAYREG have resulted in a speeded-up algorithm, reducing computer time by a factor of two to four in four benchmark examples--involving grade point average as a predictor of American College Test scores in junior colleges--believed to be fairly representative examples of the type of multiple regression problems where Bayesian m-group regression might be used. However, further use of the speeded algorithm is recommended before the program is included in the CADA Monitor. This report includes descriptions of: (1) an update to Jones and Novick's input description for use of the modified programs; (2) deficiencies of the old programs BPREP and BAYREG, and improvements in the new program, MBREG; (3) detailed proposal for a small revision of BPREP into a new preparatory program BPREP adapted to the new main program MBREG; (4) "leaps" which constitute the major performance improvement of MBREG over BAYREG. (Author/GDC)

The Feldt-Gilmer congeneric reliability coefficients make it possible to estimate the reliability of a test composed of parts of unequal, unknown length. The approximate standard errors of the Feldt-Gilmer coefficients are derived via a method using the multivariate Taylor's expansion. Monte Carlo simulation is employed to corroborate the theoretical approximations for eight hypothetical tests. The Feldt-Gilmer coefficients are shown to be appropriate with both congeneric and tau-equivalent parts. Their superiority over the special case of Kristof's coefficient increases with the number of parts. Their standard errors also compare favorably to that of Cronbach's alpha coefficient when applied to tests with tau-equivalent parts. Tables illustrate the charts. (Author/CM)